Papers with abstract formulation
Can LLMs Reason Abstractly Over Math Word Problems Without CoT? Disentangling Abstract Formulation From Arithmetic Computation (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) are often evaluated on math word problems . however, such metrics conflate two distinct sub-skills: abstract formulation and arithmetic computation. |
| Approach: | They propose to use Final-answer-based metrics to evaluate large language models on math word problems to conflate two distinct sub-skills: abstract formulation and arithmetic computation. |
| Outcome: | The proposed model performance is bottlenecked by arithmetic computation and not abstract formulation, the study shows. |